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ABSTRACT 



This report presents an evaluation of the Prichard Committee 
for Excellence's Commonwealth Institute for Parent Leadership, which trains 
Kentucky parents as informed advocates for educational excellence. The 
evaluation examined whether Institute training would increase parents' 
understanding of education reform and willingness to work for improvement; 
whether parents used their new knowledge and skills to exercise effective 
school leadership; Institute strengths and weaknesses; and whether the 
Institute works equally well in diverse communities. Data from a demographic 
examination of Institute graduates and surveys of 1998 and 1999 graduates 
highlighted five ways parents can promote change that affects students: 
accepting and communicating the message that schools must serve all students 
well; gathering data about school needs and presenting findings publicly; 
talking knowledgeably about educational issues; initiating activities that 
make schools more welcoming for parents and promote productive parent- teacher 
communication; and leveraging contacts made during training into additional 
resources for change. Organizations promoting parental involvement should 
take such actions as seeking diverse parents, providing high quality 
training, and developing reasonable outcome expectations. Evaluation 
strategies to facilitate program improvement include having ongoing 
communication between evaluators and project staff, ensuring there are no 
surprises in report findings, and building flexibility into evaluation 
design. (SM) 
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Introduction 



Evaluation is integral to our work as grantmakers. Evaluation provides the Trusts an opportu- 
nity to learn from our grantmaking and from the work of our grantees. At the most basic 
level, an evaluation can tell us whether a particular initiative met its goal. But more impor- 
tantly, evaluation helps us understand why a particular goal was or was not met. With this 
knowledge, we can work with our grantees to make mid-course corrections, as well as refine 
our grantmaking strategies. 

The Trusts have supported the Commonwealth Institute for Parent Leadership (the Insti- 
tute) — an initiative of the Prichard Committee for Academic Excellence — since 1997, based on 
our belief in the Institute's strategy of training parents as informed advocates for educational 
excellence. Our confidence in the efficacy of the Institute's model was grounded in our work 
with the Prichard Committee and our knowledge of its strong history as an advocate for 
educational excellence in Kentucky. In 1998, we decided to put our belief in the Institute to 
the test by supporting an evaluation of its work. 

Our goal for the evaluation was two-fold. First, to understand the Institute 's impact and the 
ways in which the program model — and its implementation — could be strengthened. Second, 
to facilitate a collaborative evaluation process with ongoing conversation between the Trusts, 
evaluators Dick Corbett and Bruce Wilson, and Bob Sexton and Bev Raimondo of the Prichard 
Committee. Based on our previous work with Bob and his colleagues at the Prichard Commit- 
tee, we knew they were committed to evidence-based inquiry, self-reflection and continuous 
improvement. We also understood that evaluation is most effective when both grantees and 
funders participate and understand each other's perspective. A process based on mutual 
respect and open communication helps to facilitate constructive use of both positive and 
negative evaluation data. 

We went into this evaluation with several questions: 

• Does participation in the Institute increase parents' understanding of education reform and 
their willingness to work for educational improvement in their community? 

• Do parents use their new knowledge and skills to exercise effective leadership in schools? If 
so, what challenges do they face? 

• What are the strengths of the Institute model? What areas can be improved? 

• Does the Institute work equally well in diverse communities across Kentucky? Are 
modifications in design or implementation required? 

The evaluation of the Institute is an excellent model for how evaluators, grantees and funders 
can work together. Dick and Bruce designed an iterative process that included significant 
opportunities for early, low-stakes sharing of findings with us and with the Institute's staff. 
This enabled Bob, Bev and their colleagues to increase their own learning and to make mid- 
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course corrections to strengthen their work. Their in-depth discussions also increased the 
evaluators' understanding of the Institute, and enabled a deeper and more nuanced inquiry. 
For instance, based on interviews and focus groups, the evaluators discovered that the complex 
and challenging process parents experience as they work to become change agents in schools 
may require further interventions from the Institute, including additional outreach to princi- 
pals. This finding informed the way that the Institute "seeded the ground" in the schools in 
which participants would undertake their projects. 

The trade-off in this type of collaborative evaluation is that when the grantee revises the 
program midway in response to early findings, the evaluators become part of the intervention 
rather than operating in the more traditional hands-off manner. When accountability con- 
cerns are foremost, a more distanced relationship between evaluator and grantee may be 
appropriate. But in this instance, we found that ongoing consultation between the evaluators, 
the grantee and staff at the Trusts resulted in effective utilization of evaluation findings. This 
process worked because Dick and Bruce balanced sensitivity to the grantee's perspective with a 
commitment to rigorous evaluation. And because Bob and Bev were committed to open and 
honest reflection, they never asked Dick and Bruce to pull their punches. The result was a 
strong evaluation that met our need for data on the impact of our grantmaking, while also 
providing a valuable collaborative learning process. 

We welcome this opportunity to share what we learned about structuring an effective evalua- 
tion process, as well as the more specific findings about the Institute. It is our hope that this 
information will prove useful to practitioners interested in replicating the Institute in their 
schools, as well as funders eager to learn about the impact of their investments. This publica- 
tion is part of a broader effort at the Trusts to disseminate the results of our work. As an 
institution, we have begun thinking about what we have learned that could be useful to those 
outside our own walls — especially our grantee partners and our philanthropic colleagues. 
Please visit our Web site for more "lessons learned" (http://www.pewtrusts.com). We would 
welcome feedback on this report, as well as dialogue on the role of evaluation in facilitating 
more effective philanthropy. 



Janet Kroll, Officer, Planning and Evaluation 
The Pew Charitable Trusts 



Mobilizing Parents for School Reform in Kentucky: 

The Commonwealth Institute for Parent Leadership 

Robert F. Sexton, Executive Director, Prichard Committee for Academic Excellence 
Beverly N. Raimondo, Director, Commonwealth Institute for Parent Leadership 

The Commonwealth Institute for Parent Leadership grew out of our efforts at the Prichard 
Committee for Academic Excellence. The Prichard Committee is an independent, nonpartisan, 
advocacy group whose mission is to engage the public to push for vastly improved Kentucky 
schools. 

The Prichard Committee and Kentucky School Reform 

Founded in 1983, the Prichard Committee was a major force behind the Kentucky Education 
Reform Act of 1990, which is a comprehensive, systemic effort to increase student achieve- 
ment. The Act sets academic standards and accountability, provides rewards and sanctions 
based on school progress, and pushes decisions to the school level. Also included in the Act 
are about 25 "opportunity to learn" measures, such as pre-school, extra learning time, tech- 
nology and teacher professional development. The concept behind the Act is to set expecta- 
tions and provide resources at the state level, then take action at the local level. 

With much of the Prichard Committee's agenda now a part of state law, since 1990 we have 
sought to inform the public, spur enough public support and provide enough time so the 
reform wouldn't be pulled up by the roots before it could grow. A friend once said that we 
were "buying time for the education system to ingest the reform." We wanted to ensure that 
the reforms mandated by law were implemented well and reconsidered when necessary. 

Our first challenge was to encourage schools to increase student achievement by meeting their 
goals under the Reform Act. Our second was to reach hundreds of Kentuckians, beyond our 
own volunteer Prichard Committee members, and actively engage them in the crusade to 
improve Kentucky schools. We wanted to form an "army" of concerned and knowledgeable 
citizens. 

After an extensive self-study and a strategic planning process, we decided in 1996 to reach out 
directly to individual parents and to give our reaching out a highly visible image — the Com- 
monwealth Institute for Parent Leadership. 

The Commonwealth Institute for Parent Leadership 

We know from our years of experience in Kentucky that only a rare group of teachers can 
transform their schools so that all students learn at high levels. In reality, most teachers need 
all sorts of help. The state must provide most of this help, such as policy coherence and focus, 
resources and training. 
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But help must also come from the community. That's the role of the Institute. We tell 
Institute parents that the driving idea behind student testing and releasing achievement data 
is an old-fashioned democratic concept: if people have useful information, they will take 
action. Educators, for instance, are expected to make changes in teaching practices, schedules 
or professional development to increase student achievement. Parents and citizens are ex- 
pected to digest annual data reports and react to them. If their schools are doing well, they 
applaud and support. If schools are doing poorly, parents need to complain and apply pressure. 

Shortly after the reform was enacted in 1990, it became clear that the reform's design principle 
(i.e., given proper achievement data, school-level educators would know what changes to 
make) was overly optimistic. Even with enhanced professional development, most faculty and 
principals either did not know how to respond or were not inclined to respond, sometimes 
thinking "this, too, shall pass." 

In this climate, we reasoned that educators needed encouragement and also needed a push. If 
teachers did not, or could not, react constructively to their schools' achievement data, some- 
one else should. Who better, we reasoned, than parents? Our theory was that parents who 
were knowledgeable about their schools' achievement data would encourage teachers to im- 
prove teaching practices and school organization, not acting as experts, but as friendly advi- 
sors. 

Although the parents we train and support are important to schools in and of themselves, it's 
also important to remember that the Prichard Committee's mission is to press for effective 
policy implementation to keep improving Kentucky's schools. From the start, the Institute 
was expected to support this advocacy function. 

With these considerations in mind, we designed the Institute as a program that provides a 
combination of information and skills, so that parents can focus on achievement and become 
strong advocates for all students. Through extensive statewide recruitment, Prichard Commit- 
tee staff encourage parents to apply for and commit to attend six days of training (three two- 
day sessions each fall). 

The curriculum includes information about Kentucky's standards-based education system as 
well as skills development for effective group processes, advocacy, action planning, communi- 
cations between parents and teachers, and reaching under-involved parents. We work with 
the Kentucky Association of School Councils to embed school achievement data in the curricu- 
lum and provide parents with their own school's data in an understandable format. During 
training, every parent uses these and other data gathered to identify priority needs for their 
schools. They use those needs as a basis for their volunteer project designs, which are 



reviewed by the Institute's regional staff (seven "community support coordinators" working 
out of home offices). 

After session training, the community support coordinators assist the graduates through 
individual coaching as well as regional gatherings, newsletters, regional pages on a Web site 
and ongoing communication. Maintaining this network is crucial to parents being able to 
make a difference in their schools. Often, it is their only resource and support. 

Parents are encouraged to remain a part of the Prichard Committee network for at least two 
years. During those two years, they return to their communities charged with organizing 
other parents to complete the volunteer projects they designed during their training. We 
expect these projects to have a lasting impact, to involve other parents and to improve student 
achievement. Through these projects, the Institute's graduates reach thousands of other 
parents. Since the Institute began in 1997, we have trained a diverse group of 900 parents in 
classes of 20 to 30 parents at various locations across the state. Since 1999, more parents have 
applied than could be accepted for the 200 seats available each year. In addition to the projects 
they implement, parents report increased involvement on school-based, decision-making councils 
(about 200 graduates serve on councils and many others on school and district committees), 
and many report better one-on-one communication with teachers about curriculum and achieve- 
ment issues. At least 17 now serve on local school boards, and one parent has been appointed 
to the state school board. 

Role and Impact of Evaluation 

As part of its support for the Institute, The Pew Charitable Trusts commissioned an evaluation. 
The team of Dick Corbett and Bruce Wilson began the evaluation by visiting Institute sessions 
in 1998. Already committed to an annual re-evaluation of our curriculum and training pro- 
cesses, we saw the evaluators as partners and gave their findings serious consideration. Corbett 
and Wilson also evaluated particular program components and whether our approach to in- 
creasing parent involvement can be used as a vehicle to increase student achievement. 

After the first year of evaluation work, the Institute's staff did things differently. The evalua- 
tors' September 1999 interim report helped us look at our parent participants as both learners 
and leaders. The report provided substantial evidence that we were achieving our goals with 
the curriculum, noting that "parents were nearly singular in their opinion that the training 
was effective and valuable." The evaluators told us that our high expectations for parents — 
that they would understand complex education issues— were attainable regardless of the par- 
ents' initial "qualifications" or background. The report also substantiated the importance of 
our regional coaches (the community support coordinators) and their need for enhanced 
"coaching" skills through professional development. 
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This feedback led us to make several changes. We became even more deliberate in our focus on 
student achievement by tightening the criteria for projects, ensuring that they were based on 
identified school needs and appropriate to the goals of improving achievement, increasing 
parental involvement and having a lasting impact. We increased our efforts to reach parent 
leaders in low-income and minority populations. And we realized some parents were "walking 
projects" who, because of their school environment, were unable to complete projects in two 
years, but who still contributed significantly to the conversation about student achievement 
in their schools. 

As we incorporated the evaluators' suggestions into our work, the evaluators contacted a new 
set of parents, those who had attended the 1999 Institutes. We kept Corbett and Wilson 
informed of curriculum changes, recruiting activities and implementation of volunteer projects. 
They talked to parents about their projects and gathered data to help characterize the schools 
our parents worked with. They talked to us about finding ways to capture the "walking 
project" successes as well as the formal project implementations. As a result, we developed a 
"non-project inventory," which each of our community support coordinators used in discus- 
sions with Institute parents. This gave us a much better sense of the significant activities 
parents undertook that could not be called a formal "project." 

Based on our discussions, we became more focused in our expectations of parents and felt 
confident that these high expectations were warranted. We increased our efforts to deliber- 
ately look for leaders in under-involved segments of the population, those who were most 
likely to go back to their communities and make a difference. As we accepted the 2001 
Institute participants, we contacted school principals and discussed what would be expected of 
participants from their schools so that they were ready for a different kind of parent activity. 
We invited former participants to share their successes and challenges with newcomers to 
prepare them for the reception they may encounter at their schools. As we considered parent 
applicants, we deliberately looked for "teams" to work together in schools or districts to 
increase their chances of successful projects. And we continued to refine and expand the 
"non-project inventory" to capture the wide variety of advocacy activities that participants 
undertake. 

Because they were not immersed in the day-to-day work as we were, the evaluators were able 
to identify areas needing our attention as well as successes and results we had not anticipated. 
We wanted to add to the national conversation about parents as effective advocates for school 
change and the evaluation helped us do that. There's no doubt in our minds that the 
partnership with the Trusts' evaluation team helped improve the Institute in important ways. 
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Key Lessons about the Role of Parents as Educational Change Agents 

H. Dickson Corbett 
Bruce Wilson 

The Pew Charitable Trusts funded an evaluation of the Commonwealth Institute for Parent 
Leadership. The purpose of this evaluation was to provide the Trusts and the Prichard Com- 
mittee with information concerning the Institute's implementation and impact, highlighting 
both the obstacles and facilitators of enabling parents to become educational change agents. 
Such information was intended to help the Trusts assess the Institute's value as a vehicle to 
promote parental involvement and leadership in school reform, and to determine the portabil- 
ity of the process to other settings. 

The evaluation used both quantitative and qualitative strategies to examine the reactions of 
the Institute's graduates (referred to as "Fellows") to their training, their actions in their 
respective home settings and the contextual influences on these efforts. 

The quantitative strategies, targeting the entire 1998 and 1999 cohorts, had two components: 
a demographic examination of the Fellows' schools using data provided by Kentucky officials to 
chart how representative these sites were of Kentucky's schools in general; and a survey sent 
to all of the 1998 and 1999 Fellows that sought opinions about 

• the increase in their knowledge about educational reform; 

• their comfort in applying that knowledge in conversations with others; 

• assessment of the value of what they had learned; 

• the ways in which they had been involved with schools prior to the Institute; and 

• how their schools solicited and welcomed parental involvement. 

The evaluation incorporated three qualitative data collection strategies. First, the surveys 
solicited written comments from Fellows to extend and explain their numerical responses. 
Second, Fellows from three regions, representing city, suburban and rural settings, partici- 
pated in open-ended interviews that covered (1) their motivations for becoming involved in 
the Institute; (2) activities in their schools during the academic year; (3) the facilitators of and 
obstacles to working in the schools; and (4) the support the Institute provided them to do 
their work. Third, to gain a different perspective on the impact of the Fellows' work, we talked 
with building principals to learn about the communities where these parents worked, the kinds 
of parental involvement activities that were commonly carried out and schools' reactions to 
the Fellows' activities. 

All of these data provided the opportunity to understand more thoroughly the problems and 
prospects of redefining parents' relationships with their children's schools. At this point in 
time, four cohorts into the Institute training, the majority of parents are still pushing for 
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school improvement. If anything, the Kentucky experience validates the original hypothesis 
that armed with information, skills and support, parents will work for change. At the same 
time, there seem to be some limits on the extent to which parents can do this. It was too 
much for the Institute to expect parents to have a direct, measurable impact on student 
achievement; however, it was not excessive for parents to have to demonstrate a "logical" link 
between the actions they proposed and positive benefits for students. Having to express this 
logic disciplined parents to separate what would be nice to do for a school from what would 
make the school a more effective institution of learning. 

Evidence from the evaluation suggests that there are five ways that parents can reasonably be 
expected to promote changes that affect students. 

First, it is reasonable to expect parents to accept and communicate the message to others that 
the schools must serve all students well. Thinking beyond the needs of one's own child to those 
of all children is not natural for parents, so accomplishing this is no small feat. Nevertheless, 
Fellows continually voiced to us their understanding that this is the stance they must willingly 
take with school personnel and other parents if they are to become effective change agents. 

Second, it is reasonable to expect parents to gather data about school needs and conditions, 
and to present their findings in a public forum. Kentucky parents collected information in 
several ways, such as gaining access to the schools' consolidated plans or initiating small 
surveys themselves. More importantly, perhaps. Fellows learned which questions to ask. Com- 
municating their conclusions publicly was, and is likely to remain, a difficult step for parents 
to take. However, they recognized that what they learned needed to be heard. Several 
expressed their ideas boldly, in large meetings before peers and professionals. Others did so 
much more quietly, for example, in a small report that was shared with members of a planning 
committee, school council or administrative team. Regardless, what parents thought the 
school should do needed to receive a hearing beyond the principal if it was to move forward. 
The success of a majority of parents in doing this suggests that reaching an audience other 
than the building principal is achievable. 

Third, it is reasonable to expect parents to talk knowledgeably about important educational 
issues. Being effective advocates for change meant that parents had to become "bilingual": 
they had to be able to speak with both parents and educators. The former came easily for 
them; the latter was another matter. Conversing effectively with educators required Fellows to 
overcome their own concerns about status as well as to understand the jargon that infuses so 
much of the world of education. Parents demonstrated a refreshing ability to process the 
flurry of information that the Institute passed along to them, to determine the relevance of 
pieces of this information for their own settings, and then to apply this knowledge in word and 
deed in the schools. Although stories about parents' gaining status in the eyes of school 
personnel remained sporadic in our data collection, the evidence is that maintaining constant 



and knowledgeable contact with the schools will ultimately result in parents having a substan- 
tive, as opposed to just a symbolic, place at schools' decision-making tables. 

Fourth, it is reasonable to expect that parents will initiate activities that make schools more 
welcoming for parents, better prepare parents to work with their children academically, and 
promote productive communication between parents and educators that leads to new programs 
and practices that benefit students' academic success. The Institute norm was that Fellows' 
projects that had a direct connection to promoting student achievement were valued more 
than ones that did not. That expectation invoked a certain level of rigor into the Fellows' 
actions and infused the Institute with an "edge" that said just becoming more involved in a 
school was not good enough. Our discussions with parents indicated that this standard was 
probably too high for some parents in some settings, and it may be that more welcoming 
schools and more academically-attuned parents are preludes to enacting collaborative, aca- 
demically-beneficial projects. 

Fifth, it is reasonable to expect that parents will leverage the contacts they make during training 
into additional resources for change. Through the Institute, parents became acquainted with a 
host of agencies, people and materials that could help them and their schools do a better job of 
serving children and their parents. It was evident that for some schools the Fellows were the 
first adults in the community to learn about and interact with potentially helpful people and 
agencies at the state level. Parents gained confidence from their initial encounters with these 
entities and became emboldened to reach out in multiple directions to garner information, 
money and influence that might further their efforts in the schools. 

Implications for Organizations Promoting Parental Involvement 

Although the above lessons might suggest that parents can play a modest role in the change 
process at best, even these reasonable expectations meant that the parents involved in the 
Institute far exceeded the kinds of actions they had previously taken in schools. Thus, for 
them, being an Institute Fellow represented a radical departure. Organizations hoping to 
encourage parents to take such steps must be intentional in organizing their efforts to meet 
this goal if participants are to have widespread success in their endeavors. Such organizations 
should: 

Seek diverse, active parents with whom to work. The staff of the Institute specifically sought 
people who were already leaders in their local schools, were involved with other community 
associations and/or supported educational reform. While using these criteria meant that the 
Institute would not tap into the pool of uninvolved and/or invisible parents, the staff and 
Fellows felt strongly that those selected would be in the best position to use the skills and 
knowledge the Institute offered. Given the expectations for greater leadership that the Insti- 
tute had for parents, this selection strategy seemed reasonable and advisable for similar pro- 
grams. Continued recruitment of this sort will be necessary in the same sites over time, as 
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parents follow their children from one building to the next. This natural transience means 
that a program must rebuild its cadre of parents even as it seeks to expand it to new schools 
and communities. 

Invest in high quality training. Each year. Fellows attended three two-day sessions over the 
course of several months in the fall. Spread across Kentucky's eight geographic regions, the 
workshops introduced Fellows to their rights to know about and gain access to school opera- 
tions, key elements of the state's reform legislation and policy, where to go and whom to 
contact for information about educational and community resources, specific ways to act as 
advocates for school reform, as well as topics related to curriculum, instruction, assessment 
and learning. The Fellows said that the information they received was eye-opening and di- 
rectly relevant to their situations, and that the manner in which the information was pre- 
sented was engaging and professionally done. 

In the interviews. Fellows continually referred to what they had learned as a primary outcome 
of their training as well as a new-found assurance that they could make a meaningful contri- 
bution to their children's schools beyond their culinary skills and a yearning to actually make 
a difference. In the Fellows' comments on the survey, 80 of the respondents (or nearly half of 
the number who returned surveys) volunteered references to what they had gained from their 
participation in the Institute. More than 95 percent of these specifically mentioned a better 
understanding of schools, greater confidence in approaching school staff and other parents 
about addressing school needs, and a willingness to spur schools to make changes. The Fellows 
frequently expressed pride and amazement in having acquired these attributes and identified 
the Institute training as the cause. 

Develop reasonable expectations for program outcomes. The three outcomes just mentioned 
reinforced one another in a "virtuous cycle." Knowledge and confidence were the foundations 
for action. Successful action then required more information, which, when acquired, boosted 
confidence and a willingness to act further. Filtering into this cycle was the fact that parents 
were encouraged to engage in this work on behalf of all children, not just their own. 

Knowledge, confidence and willingness were the primary indicators of the program's success. 
In truth, these fell short of the Institute's original intention which was to have the parents 
directly and measurably affect student achievement. In practice, it became apparent that it 
was most reasonable to expect parents to take actions that had a logical, rather than a causal, 
connection to student achievement, mostly because it is statistically impossible to tease out 
the relative effects of a single initiative on student learning. The concept of student achieve- 
ment, therefore, should discipline participants' thinking about what they should do in their 
schools. However, its measurement should not be used to judge the effectiveness of those 
actions. 



Prepare parents and key staff to develop a new relationship in the schools in which they will 
work. Just as parents were not accustomed to acting as change agents, school staff were 
mostly unprepared for working with parents who were attempting to enact this role. While it 
may be unreasonable to expect an organization supporting Institute-like Fellows to offer corre- 
sponding training for school staff, it is necessary for the organization to at least inform the 
school about the training the parent is receiving and the kinds of activities the parent might 
want to organize. General communications to central offices seemed ineffective in most of the 
communities. Thus, direct contact with each principal is more advisable. 

Surround parent participants with support during the school year. This was accomplished 
effectively in the Institute in two ways: creating the position of Community Support Coordina- 
tor (CSC) and pairing parents in the same community or school. The Institute used regionally- 
based CSCs to serve as resources to the Fellows as they attempted to engage schools. The 
ongoing availability of the CSCs to the Fellows earned considerable praise and admiration. The 
CSCs provided on-demand individual assistance with project design, arranged numerous peri- 
odic meetings among groups of Fellows, and attended or facilitated Fellows' activities in the 
schools. The CSCs also served as the main link to new information about the Institute via 
newsletters, regular mailings, electronic mail and telephone contact. In other words, the CSCs 
were the lifeline to the initiative. But more important than the materials they provided was 
the moral support. Fellows knew there was always someone they could count on for assis- 
tance. And they argued that any program attempting to copy the Institute had to include 
provisions for this type of ongoing communication and assistance. 

Initially, few parents were drawn from the same schools in an attempt to broaden the program's 
reach. Over time, however, as several parents from the same communities and even the same 
buildings began to talk with one another, they realized the value of being able to confer, 
commiserate and collaborate with other parents. Institute staff realized that this somewhat 
serendipitous development should become an intentional part of the program's design and 
began to recruit parent pairs in later cohorts. This strategy of pairing parents should become 
a part of the initial design of programs seeking to emulate the Institute. 

Celebrate parents' successes. Parents needed recognition for their efforts, both to acknowledge 
what they accomplished and to let other parents know what was possible for them to do. 
Examples abounded. The Institute regularly invited veterans to annual meetings to share 
stories about their experiences. The CSCs and the Institute staff often attended parent- 
organized events in their communities, which lent an air of importance to them, and the state 
newsletter contained descriptions of Fellows' actions. All of these developments validated that 
parents' efforts were widely valued and provided a source of inspiration for further actions 
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Evaluation Strategies to Facilitate Program Improvement 

In addition to supplying information about a program's eventual effectiveness, an evaluation 
should serve a formative function to improve program operation along the way. Doing so 
means that the evaluation promotes reflection and adjustment, not just at the end of the 
evaluation period but also during it. This requires that evaluators offer periodic feedback to 
the project under conditions that make it possible for the project staff to interpret the findings 
and to brainstorm alternative avenues of action. 

There are at least three strategies that maximize the possibility that evaluators' information 
will receive a productive and constructive hearing. The first is having ongoing communication 
between the evaluators and project staff. This partially entails periodic meetings with program 
staff to review the evaluation design, to see how this design needs to be adjusted to accommo- 
date other demands on participants' time and energy, to take new developments in the pro- 
gram into account, and to support particular information needs that may fall within the 
parameters of the evaluation. Communication should also include reminders of what and when 
evaluation activities will take place. 

Such communication benefits all parties. For example. Institute staff reviewed survey items 
prior to their use with parents and made suggestions about including items of importance to 
them that fell within the boundaries of the evaluation. In return, they made several pleas to 
Fellows to complete the surveys as part of their commitment to the project. This heightened 
the quantity and quality of the data as well as the Institute's interest in the results. During 
the two years of the evaluation, all were kept abreast of where the evaluation was in terms of 
the overall design and when they might expect to hear that the evaluators had been contact- 
ing participants. 

The second strategy is making sure that there are no "surprises" in report findings. Any 
conclusions and/or recommendations contained in interim and final reports should be dis- 
cussed ahead of the reports being written. As in all human interaction, words have multiple 
meanings. Actually talking to program staff prior to finalizing the report increases the prob- 
ability that staff will understand what the evaluators are saying and provides the evaluators 
with other ways of looking at the data they have collected. This process should not be 
construed as allowing staff to write the evaluation report, but rather as increasing the possibil- 
ity that the report will receive a fair and thorough hearing. Evaluators talked with Institute 
staff in person prior to writing both the interim and final reports and, in each case, found that 
staff had already made changes in their approach by the time they had received their final 
copies. 

This strategy also ensures that the evaluation fits the program's reflection cycle. As an 
example, the Institute's work with each parent cohort essentially corresponded to the school 
year. The reporting cycle for the evaluation ran from fall to fall. Were the information 
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contained in the reports not available to staff until mid-fall, staff would have been too far into 
their Institute to utilize the information. 

The final strategy is having flexibility built into the evaluation design so that it can adapt to 
changes in the program along the way. The Institute evaluation was originally intended to 
focus on a single cohort in an attempt to make sure that there were enough resources to 
unearth the richness of parents' actions in their communities. However, Prichard staff were 
adept and eager to fine-tune the training and their interaction with Fellows. This suggested 
that the evaluation could be most helpful by tracing out the effects of these adaptations across 
a couple of cohorts. This need, coupled with the fact that the surveys (particularly the 
willingness of Fellows to add written comments to them) and interviews were yielding the 
desired depth of information, made it possible to alter the evaluation so that it better suited 
changes in the program's operation. Having the opportunity to enact a flexible evaluation 
design was a function of regular communication among Prichard, The Pew Charitable Trusts 
and the evaluators. 

Essentially, then, an evaluation is most helpful when its findings are presented against the 
backdrop of a healthy relationship with program and funding agency staff. This allows evalu- 
ators to play the role of a "critical friend" — someone sympathetic to the program's goals and 
informed enough about program actions to make valid observations about its progress toward 
attaining them. 
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For additional information: 



The Prichard Committee for Academic Excellence 
Bob Sexton, Executive Director 
http://www.prichardcommittee.org 
P.O. Box 1658 

Lexington, Kentucky 40588-1658 
Telephone: 859.233.9849 
Toll-free: 800.928.2111 
Facsimile: 859.233.0760 
E-mail: admin@prichardcommittee.org 

The Commonwealth Institute for Parent Leadership 
Bev Raimondo, Director 
http://www.cipl.org 
P.O. Box 1658 

Lexington, Kentucky 40588-1658 
Telephone: 859.233.9849 
Toll-free: 800.928.2111 
Facsimile: 859.233.0760 
E-mail: cipl@prichardcommittee.org 

H. Dickson Corbett 
512 Conestoga Road 
Malvern, PA 19355 
Telephone: 610.408.9206 
Facsimile: 610.408.8268 
E-mail: corbett@pond.com 

Bruce L. Wilson 
11 Linden Avenue 
Merchantville, NJ 08109 
Telephone: 856.662.6424 
Facsimile: 856.662.6434 
Email: bwilson@voicenet.com 

Janet Kroll 

Officer, Planning and Evaluation 
The Pew Charitable Trusts 
2005 Market Street, Suite 1700 
Philadelphia, PA 19103-7077 
Telephone: 215.575.4756 
Facsimile: 215.575.4888 
E-mail: jkroll@pewtrusts.com 
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